FIX: Align date/time type code mappings with ODBC 18 driver source (#352) #355

dlevy-msft-sql · 2025-12-01T17:19:35Z

Work Item / Issue Reference

GitHub Issue: #352

Summary

Fixes polars.read_database() ComputeError on DATE columns by aligning cursor.py type code mappings with the ODBC 18 driver's actual reported SQL type codes.

Root Cause:

The ODBC 18 driver reports ODBC 3.x type codes (SQL_TYPE_DATE=91, SQL_TYPE_TIMESTAMP=93, SQL_SS_TIME2=-154) but _map_data_type only had ODBC 2.x constants (SQL_DATE=9, SQL_TIME=10, SQL_TIMESTAMP=11). DATE columns fell through to the str default, causing a schema mismatch when polars expected pl.Date.

Fix (verified against ODBC 18 driver C++ source):

_map_data_type: Rewritten with driver-verified ODBC 3.x type codes organized by category (string, integer, float, decimal, date/time, boolean, binary, UUID, XML). Removed phantom ODBC 2.x entries that the driver never reports.
_get_c_type_for_sql_type: Updated C-type bindings from ODBC 2.x to 3.x codes for all date/time types.
constants.py: Added SQL_SS_TIME2, SQL_SS_XML, SQL_C_SS_TIME2.

Zero breaking changes. cursor.description[i][1] continues to return Python type objects (datetime.date, datetime.datetime, etc.) — now mapped from the correct SQL type codes.

Changes

File	Change
`mssql_python/constants.py`	+3 constants (`SQL_SS_TIME2`, `SQL_SS_XML`, `SQL_C_SS_TIME2`)
`mssql_python/cursor.py`	Rewritten `_map_data_type` and `_get_c_type_for_sql_type` with ODBC 3.x codes
`tests/test_018_polars_pandas_integration.py`	NEW — 14 integration tests

Testing

14 new tests across 3 test classes:

TestCursorDescriptionTypeCodes (7 tests): Verifies all 6 date/time SQL types return correct Python types with isclass() checks
TestPolarsIntegration (4 tests): polars.read_database() with DATE, all datetime types, mixed types, NULLs
TestPandasIntegration (3 tests): pandas.read_sql() with DATE, all datetime types, mixed types

All 14 tests pass. Existing test suite (448 tests) unaffected.

Copilot

Pull request overview

This PR fixes issue #352 by implementing two main improvements: (1) correcting cursor.description to return SQL type codes (integers) instead of Python type objects, ensuring DB-API 2.0 specification compliance, and (2) adding support for SQL Server spatial data types (geography, geometry, hierarchyid) by handling the SQL_SS_UDT type code (-151).

Key changes:

Fixed cursor.description[i][1] to return SQL type integer codes (e.g., 4, -9) instead of Python types (int, str) per DB-API 2.0 spec
Added SQL_SS_UDT (-151) support for SQL Server User-Defined Types including spatial data types
Updated output converter lookup to use SQL type codes consistently throughout the codebase

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
mssql_python/cursor.py	Changed `cursor.description` to return SQL type codes instead of Python types; added `_column_metadata` storage; updated `_build_converter_map` to use SQL type codes; added mappings for UDT, XML, DATETIME2, SMALLDATETIME types
mssql_python/constants.py	Added `SQL_SS_UDT = -151` constant for SQL Server User-Defined Types
mssql_python/pybind/ddbc_bindings.cpp	Added C++ constants for SQL_SS_UDT, SQL_DATETIME2, SQL_SMALLDATETIME; implemented UDT handling in SQLGetData_wrap, FetchBatchData, FetchMany_wrap, and FetchAll_wrap for LOB streaming
tests/test_004_cursor.py	Added `lob_wvarchar_column` to test schema; updated `test_cursor_description` to expect SQL type codes; added comprehensive geography type test suite (14 new tests); separated LOB and non-LOB fetch tests; fixed output converter test for UTF-16LE encoding
tests/test_003_connection.py	Simplified converter integration tests to use SQL type constants directly instead of dynamic type detection

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

tests/test_004_cursor.py

mssql_python/constants.py

mssql_python/cursor.py

github-actions · 2026-01-05T19:27:13Z

📊 Code Coverage Report

🔥 Diff Coverage 100%	🎯 Overall Coverage 76%	📈 Total Lines Covered: `5475` out of `7139` 📁 Project: `mssql-python`

Diff Coverage

Diff: main...HEAD, staged and unstaged changes

mssql_python/constants.py (100%)

Summary

Total: 2 lines
Missing: 0 lines
Coverage: 100%

📋 Files Needing Attention

📉 Files with overall lowest coverage (click to expand)

mssql_python.pybind.logger_bridge.hpp: 58.8%
mssql_python.pybind.logger_bridge.cpp: 59.2%
mssql_python.row.py: 66.2%
mssql_python.pybind.ddbc_bindings.cpp: 69.3%
mssql_python.pybind.ddbc_bindings.h: 69.7%
mssql_python.pybind.connection.connection.cpp: 75.3%
mssql_python.ddbc_bindings.py: 79.6%
mssql_python.pybind.connection.connection_pool.cpp: 79.6%
mssql_python.connection.py: 84.1%
mssql_python.cursor.py: 84.7%

🔗 Quick Links

⚙️ Build Summary	📋 Coverage Details
View Azure DevOps Build	Browse Full Coverage Report

mssql_python/cursor.py

Copilot

Pull request overview

Copilot reviewed 16 out of 17 changed files in this pull request and generated 6 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

mssql_python/cursor.py

tests/test_004_cursor.py

Copilot

Pull request overview

Copilot reviewed 16 out of 17 changed files in this pull request and generated no new comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot

Pull request overview

Copilot reviewed 16 out of 17 changed files in this pull request and generated no new comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

mssql_python/cursor.py

Copilot

Pull request overview

Copilot reviewed 16 out of 17 changed files in this pull request and generated 3 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

mssql_python/pybind/ddbc_bindings.cpp

mssql_python/cursor.py

mssql_python/constants.py

- Add SQL_SS_XML (-152) constant to Python constants.py (was incorrectly using SQL_XML = 241) - Add SQL_SS_TIME2 (-154) constant for SQL Server TIME(n) type - Update cursor type map to use SQL_SS_XML and add SQL_SS_TIME2 mapping - Add sync comment in C++ to prevent future constant drift Constants verified against Microsoft Learn ODBC documentation: - SQL_SS_TIME2: -154 (SQLNCLI.h) - SQL_SS_TIMESTAMPOFFSET: -155 (SQLNCLI.h) - SQL_SS_XML: -152 (SQL Server ODBC driver) - SQL_SS_UDT: -151 (SQL Server ODBC driver) Addresses Copilot review feedback on PR microsoft#355

Copilot

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

mssql_python/constants.py

tests/test_018_polars_pandas_integration.py

Copilot

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

mssql_python/cursor.py

tests/test_018_polars_pandas_integration.py

Copilot

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

mssql_python/constants.py

Copilot

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

tests/test_018_polars_pandas_integration.py

Copilot

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

tests/test_018_polars_pandas_integration.py

Copilot

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

mssql_python/cursor.py

Root cause: _map_data_type only had ODBC 2.x constants (SQL_DATE=9, SQL_TIME=10, SQL_TIMESTAMP=11) but the ODBC 18 driver reports ODBC 3.x codes via SQLDescribeCol. Date columns returned SQL_TYPE_DATE(91) which fell through to str default, causing polars ComputeError. Verified against ODBC 18 driver source (sqlcmisc.cpp rgbSRV2SQLTYPE[] and sqlcdesc.cpp SQL_DESC_CONCISE_TYPE handling): constants.py: - Add SQL_SS_TIME2(-154), SQL_SS_XML(-152), SQL_C_SS_TIME2(0x4000) cursor.py _map_data_type: - Replace ODBC 2.x entries with driver-verified ODBC 3.x codes - SQL_TYPE_DATE(91) -> datetime.date - SQL_TYPE_TIMESTAMP(93) -> datetime.datetime - SQL_SS_TIME2(-154) -> datetime.time - SQL_DATETIMEOFFSET(-155) -> datetime.datetime - SQL_SS_XML(-152) -> str - Add missing types: SQL_LONGVARCHAR, SQL_WLONGVARCHAR, SQL_REAL cursor.py _get_c_type_for_sql_type: - SQL_TYPE_DATE -> SQL_C_TYPE_DATE (was SQL_DATE) - SQL_SS_TIME2 -> SQL_C_SS_TIME2 (was SQL_TIME) - SQL_TYPE_TIMESTAMP -> SQL_C_TYPE_TIMESTAMP (was SQL_TIMESTAMP) - Add SQL_DATETIMEOFFSET -> SQL_C_SS_TIMESTAMPOFFSET Tests: - 14 tests: cursor.description type_code verification (6 date/time types + isclass check), polars integration (4), pandas integration (3) Closes microsoft#352

Copilot

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

dlevy-msft-sql · 2026-02-11T03:40:55Z

I had to modify some code on my side to handle this change but now the date is properly recognized so this fixes #425 for me.

Did you have to change anything unexpected or was it a matter of coding it like you would code pyodbc?

sdebruyn/dbt-fabric@2ee57d2

Yes and no... I think the original implementation was a bit odd.

I had the diagnosis wrong and went a long way down the wrong road. The fix was actually simple. It should work without any code change for you now.

Copilot AI review requested due to automatic review settings December 1, 2025 17:19

Copilot started reviewing on behalf of dlevy-msft-sql December 1, 2025 17:20 View session

Copilot finished reviewing on behalf of dlevy-msft-sql December 1, 2025 17:23

Copilot AI reviewed Dec 1, 2025

View reviewed changes

tests/test_004_cursor.py Show resolved Hide resolved

tests/test_004_cursor.py Outdated Show resolved Hide resolved

mssql_python/constants.py Outdated Show resolved Hide resolved

mssql_python/cursor.py Outdated Show resolved Hide resolved

dlevy-msft-sql added bug Something isn't working pr-size: medium Moderate update size labels Dec 1, 2025

dlevy-msft-sql force-pushed the Issue-352 branch from 3f52dd7 to 646ef04 Compare December 9, 2025 20:56

dlevy-msft-sql added pr-size: small Minimal code update pr-size: medium Moderate update size and removed pr-size: medium Moderate update size labels Jan 16, 2026

sumitmsft reviewed Jan 19, 2026

View reviewed changes

mssql_python/cursor.py Outdated Show resolved Hide resolved

sumitmsft reviewed Jan 19, 2026

View reviewed changes

mssql_python/cursor.py Outdated Show resolved Hide resolved

sumitmsft reviewed Jan 19, 2026

View reviewed changes

mssql_python/cursor.py Show resolved Hide resolved

dlevy-msft-sql requested a review from Copilot January 19, 2026 19:48

Copilot started reviewing on behalf of dlevy-msft-sql January 19, 2026 19:48 View session

Copilot AI reviewed Jan 19, 2026

View reviewed changes

mssql_python/cursor.py Show resolved Hide resolved

tests/test_004_cursor.py Outdated Show resolved Hide resolved

tests/test_004_cursor.py Show resolved Hide resolved

tests/test_004_cursor.py Show resolved Hide resolved

tests/test_004_cursor.py Show resolved Hide resolved

dlevy-msft-sql requested a review from Copilot January 19, 2026 20:10

Copilot started reviewing on behalf of dlevy-msft-sql January 19, 2026 20:10 View session

Copilot AI reviewed Jan 19, 2026

View reviewed changes

dlevy-msft-sql requested a review from Copilot January 19, 2026 20:18

Copilot started reviewing on behalf of dlevy-msft-sql January 19, 2026 20:18 View session

Copilot AI reviewed Jan 19, 2026

View reviewed changes

dlevy-msft-sql linked an issue Jan 19, 2026 that may be closed by this pull request

Cursor.description returning incorrect values #352

Open

sumitmsft reviewed Jan 20, 2026

View reviewed changes

mssql_python/cursor.py Show resolved Hide resolved

dlevy-msft-sql requested a review from Copilot January 20, 2026 05:13

Copilot started reviewing on behalf of dlevy-msft-sql January 20, 2026 05:13 View session

Copilot AI reviewed Jan 20, 2026

View reviewed changes

mssql_python/pybind/ddbc_bindings.cpp Outdated Show resolved Hide resolved

mssql_python/cursor.py Outdated Show resolved Hide resolved

mssql_python/constants.py Outdated Show resolved Hide resolved

dlevy-msft-sql requested a review from Copilot January 20, 2026 05:38

dlevy-msft-sql requested a review from Copilot February 10, 2026 17:45

Copilot started reviewing on behalf of dlevy-msft-sql February 10, 2026 17:46 View session